fix(span): track unnormalized source len for dep-info #148962

weihanglo · 2025-11-15T05:14:21Z

Add unnormalized_source_len field to track the byte length
of source files before normalization (the original length).

unnormalized_source_len is for writing the correct file length
to dep-info for -Zchecksum-hash-algorithm

Fixes #148934

rustbot · 2025-11-15T05:14:26Z

r? @jieyouxu

rustbot has assigned @jieyouxu.
They will have a look at your PR within the next two weeks and either review your PR or reassign to another reviewer.

Use r? to explicitly pick a reviewer

weihanglo · 2025-11-15T05:15:57Z

compiler/rustc_span/src/lib.rs

        self.checksum_hash.encode(s);
        // Do not encode `start_pos` as it's global state for this session.
        self.source_len.encode(s);
+        self.original_source_len.encode(s);


Not sure if this is really needed

I would say if the normalized source len is encoded, then unnormalized source len should also be encoded. I would imagine that if the source file changed, then well, it did materially change.

weihanglo · 2025-11-15T05:41:56Z

compiler/rustc_interface/src/passes.rs

                (
                    escape_dep_filename(&fmap.name.prefer_local().to_string()),
-                    fmap.source_len.0 as u64,
+                    fmap.original_source_len as u64,


This line is the goal of the entire PR

Suggestion: can you please add a comment to elaborate here for locality? That is,

source_len -> original_source_len is very subtle.

Added. Yeah it is clearer now.

jieyouxu

I'm not an expert in this part of the compiler, however the changes make sense to me. A few "cosmetic" nits.

View changes since this review

jieyouxu · 2025-11-15T08:40:00Z

compiler/rustc_interface/src/passes.rs

                (
                    escape_dep_filename(&fmap.name.prefer_local().to_string()),
-                    fmap.source_len.0 as u64,
+                    fmap.original_source_len as u64,


Suggestion: can you please add a comment to elaborate here for locality? That is,

source_len -> original_source_len is very subtle.

jieyouxu · 2025-11-15T08:43:18Z

compiler/rustc_span/src/lib.rs

+    /// The byte length of this source before normalization.
+    pub original_source_len: u32,


Suggestion: I'm not sure how invasive the change is, but we can rename this pair of source lengths (better names welcome):

source_len -> normalized_source_len

original_source_len -> unnormalized_source_len

When reading this diff, I find that source_len vs original_source_len is really not obvious. I can appreciate if the diffs is intentionally made small to make review easier, but I would prefer if we also do a rename for the source_len field for consistency -- this is really not obvious.

Nice suggestion. Done in a separate commit.

jieyouxu · 2025-11-15T08:48:16Z

compiler/rustc_span/src/lib.rs

        self.checksum_hash.encode(s);
        // Do not encode `start_pos` as it's global state for this session.
        self.source_len.encode(s);
+        self.original_source_len.encode(s);


I would say if the normalized source len is encoded, then unnormalized source len should also be encoded. I would imagine that if the source file changed, then well, it did materially change.

jieyouxu · 2025-11-15T08:50:27Z

@rustbot author

jieyouxu · 2025-11-15T14:01:19Z

Fun. clippy is another josh subtree, so it's okay to change r-l/r side since we obviously need to change the compiler tree here.

weihanglo · 2025-11-15T14:03:29Z

At least Cargo doesn't have that, otherwise I'll need to do two more PRs 😬

weihanglo · 2025-11-15T14:04:18Z

Oops typo

This is a preparation for introducing a unnormalized source length field

Add `unnormalized_source_len` field to track the byte length of source files before normalization (the original length). `unnormalized_source_len` is for writing the correct file length to dep-info for `-Zchecksum-hash-algorithm`

rustbot · 2025-11-15T14:05:28Z

Some changes occurred in src/tools/clippy

cc @rust-lang/clippy

jieyouxu

Thanks

View changes since this review

jieyouxu · 2025-11-16T07:41:09Z

@bors r+ rollup

bors · 2025-11-16T07:41:12Z

📌 Commit cf57b9b has been approved by jieyouxu

It is now in the queue for this repository.

fix(span): track unnormalized source len for dep-info Add `unnormalized_source_len` field to track the byte length of source files before normalization (the original length). `unnormalized_source_len` is for writing the correct file length to dep-info for `-Zchecksum-hash-algorithm` Fixes rust-lang#148934

Rollup of 3 pull requests Successful merges: - #145954 (stabilize extern_system_varargs) - #148962 (fix(span): track unnormalized source len for dep-info) - #148969 (compiletest: Don't apply "emscripten" directives to `wasm32-unknown-unknown`) r? `@ghost` `@rustbot` modify labels: rollup

rustbot assigned jieyouxu Nov 15, 2025

weihanglo commented Nov 15, 2025

View reviewed changes

weihanglo mentioned this pull request Nov 15, 2025

-Zchecksum-hash-algorithm used normalized file size in dep-info #148934

Open

weihanglo commented Nov 15, 2025

View reviewed changes

jieyouxu reviewed Nov 15, 2025

View reviewed changes

rustbot added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Nov 15, 2025

weihanglo force-pushed the checksum branch from 00dcb1d to b55a79a Compare November 15, 2025 13:40

weihanglo changed the title ~~fix(span): track original source len for dep-info~~ fix(span): track unnormalized source len for dep-info Nov 15, 2025

weihanglo requested a review from jieyouxu November 15, 2025 13:44

rustbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels Nov 15, 2025

This comment has been minimized.

Sign in to view

weihanglo force-pushed the checksum branch from b55a79a to ddba458 Compare November 15, 2025 14:03

weihanglo added 2 commits November 15, 2025 09:05

refactor(span): rename source_len to normalized_source_len

5f38582

This is a preparation for introducing a unnormalized source length field

fix(span): track unnormalized source len for dep-info

cf57b9b

Add `unnormalized_source_len` field to track the byte length of source files before normalization (the original length). `unnormalized_source_len` is for writing the correct file length to dep-info for `-Zchecksum-hash-algorithm`

weihanglo force-pushed the checksum branch from ddba458 to cf57b9b Compare November 15, 2025 14:05

rustbot added the T-clippy Relevant to the Clippy team. label Nov 15, 2025

jieyouxu approved these changes Nov 16, 2025

View reviewed changes

bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Nov 16, 2025

Zalathar mentioned this pull request Nov 16, 2025

Rollup of 3 pull requests #148992

Open

		/// The byte length of this source before normalization.
		pub original_source_len: u32,

fix(span): track unnormalized source len for dep-info #148962

Are you sure you want to change the base?

fix(span): track unnormalized source len for dep-info #148962

Conversation

weihanglo commented Nov 15, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

rustbot commented Nov 15, 2025

Uh oh!

weihanglo Nov 15, 2025

Choose a reason for hiding this comment

Uh oh!

jieyouxu Nov 15, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

weihanglo Nov 15, 2025

Choose a reason for hiding this comment

Uh oh!

jieyouxu Nov 15, 2025

Choose a reason for hiding this comment

Uh oh!

weihanglo Nov 15, 2025

Choose a reason for hiding this comment

Uh oh!

jieyouxu left a comment • edited by rustbot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jieyouxu Nov 15, 2025

Choose a reason for hiding this comment

Uh oh!

jieyouxu Nov 15, 2025

Choose a reason for hiding this comment

Uh oh!

weihanglo Nov 15, 2025

Choose a reason for hiding this comment

Uh oh!

jieyouxu Nov 15, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jieyouxu commented Nov 15, 2025

Uh oh!

This comment has been minimized.

jieyouxu commented Nov 15, 2025

Uh oh!

weihanglo commented Nov 15, 2025

Uh oh!

weihanglo commented Nov 15, 2025

Uh oh!

rustbot commented Nov 15, 2025

Uh oh!

jieyouxu left a comment • edited by rustbot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jieyouxu commented Nov 16, 2025

Uh oh!

bors commented Nov 16, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

weihanglo commented Nov 15, 2025 •

edited

Loading

jieyouxu Nov 15, 2025 •

edited

Loading

jieyouxu left a comment •

edited by rustbot

Loading

jieyouxu Nov 15, 2025 •

edited

Loading

jieyouxu left a comment •

edited by rustbot

Loading